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Abstract 



We consider the problem of testing for the presence (or detection) of an 
unknown sparse signal in additive white noise. Given a fixed measurement 

^^ budget, much smaller than the dimension of the signal, we consider the 

general problem of designing compressive measurements to maximize the 
measurement signal-to- noise ratio (SNR), as increasing SNR improves the 
detection performance in a large class of detectors. We use a lexicographic 
Y^ optimization approach, where the optimal measurement design for sparsity 

level k is sought only among the set of measurement matrices that satisfy 
the optimality conditions for sparsity level k — 1. We consider optimizing two 
J> different SNR criteria, namely a worst-case SNR measure, over all possible 

realizations of a fc-sparse signal, and an average SNR measure with respect 
to a uniform distribution on the locations of the up to k nonzero entries in 
the signal. We establish connections between these two criteria and certain 

f — , classes of tight frames. We constrain our measurement matrices to the 

class of tight frames to avoid coloring the noise covariance matrix. For the 
worst-case problem, we show that the optimal measurement matrix is a 
Grassmannian line packing for most — and a uniform tight frame for all — 

. ,-h sparse signals. For the average SNR problem, we prove that the optimal 

^S measurement matrix is a uniform tight frame with minimum sum- coherence 

C^ for most — and a tight frame for all — sparse signals. 
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1. Introduction 

Over the past few years, considerable progress has been made towards 
developing a mathematical framework for reconstructing sparse or compress- 
ible signals)^] The most notable result is the development of the compressed 
sensing theory (see, e.g., [I]-[6]), which shows that an unknown signal can 
be recovered from a small (relative to its dimension) number of linear mea- 
surements provided that the signal is sparse. Thus, compressed sensing and 
related sparse recovery methods have become topics of great interest, leading 
to many exciting developments in sparse representation theory, measurement 
design, and sparse recovery algorithms (see, e.g, [7]-[15]). 

The major part of the effort, however, has been focused on estimating 
sparse signals. Hypothesis testing (detection and classification) involving 
sparse signal models, on the other hand, has been scarcely addressed, no- 
table exceptions being |16j-|20j. Detecting a sparse signal in noise is fun- 
damentally different from reconstructing a sparse signal, as the objective in 
detection often is to maximize the probability of detection or to minimize 
a Bayes risk, rather than to find the sparsest signal that satisfies a linear 
observation equation. We note that in the compressed sensing literature 
the term "sparse signal detection" often means identifying the support of a 
sparse signal. In this paper, however, we use this term to refer to a binary 
hypothesis test for the presence or absence of a sparse signal in noise. The 
problem is to decide whether a measurement vector is a realization from 
a hypothesized noise only model or from a hypothesized signal-plus-noise 
model, where in the latter model the signal is sparse in a known basis but 
the indices and values of its nonzero coordinates are unknown. 

Existing work (e.g., see |16j-|18j) is mainly focused on understanding 
how the performance of well-known detectors (e.g., the Neyman-Pearson 
detector) are affected by measurement matrices that have the so-called re- 
stricted isometry property (RIP). The RIP condition for the measurement 
matrix is sufficient for the minimum ^i-norm solution to be exact (or near- 
exact when the measurements are noisy) (e.g., see [1]). A fundamental result 



X A vector x = [xi,X2, . . ■ , £jv] t is sparse when the cardinality of its support S = {k : 
Xk 7^ 0} is much smaller than its dimension N. A vector x is compressible, if its entries 
obey a power law, i.e., the fcth largest entry in absolute value, denoted by |x|(fc), satisfies 
M(fc) — CV ■ k~ r , r > 1 and C r is a constant depending only on r (see, e.g., [T]). Then 
||x — Xfe||i < VkC' r ■ k~ r+1 , where Xfe is the best fc-term approximation of x. 



of compressed sensing has been to establish that random matrices, with in- 
dependently and identically distributed (i.i.d.) Gaussian or i.i.d. Bernoulli 
entries, satisfy the RIP condition with high probability. The analysis pre- 
sented in [16] and [T7] provides theoretical bounds on the performance of a 
Neyman-Pearson detector — quantified by the maximum probability of de- 
tection achieved at a pre-specified false alarm rate — when matrices with i.i.d. 
Gaussian entries are used for collecting measurements. In |18j . the authors 
derive bounds on the total error probability for detection, involving both 
false alarm and miss detection probabilities, but again for measurement 
matrices with i.i.d. entries. Finally, in |19j and [20], the authors develop 
compressive matched subspace detectors that also use random matrices for 
collecting measurements for detecting sparse signals in known subspaces. 

The body of work reported in PU]-[2n] provides a valuable analysis of 
the performance of different detectors, but leave the question of how to de- 
sign measurement matrices to optimize a measure of detection performance 
open. As in the case of reconstruction, random matrices have been studied 
in these papers in the context of signal detection primarily because of the 
tractability of the associated performance analysis. But what are the neces- 
sary and sufficient conditions a compressive measurement matrix must have 
to optimize a desired measure of detection performance? How can matrices 
that satisfy such conditions be constructed? Our aim in this paper is to take 
initial but significant steps towards answering these questions. We further 
clarify our goals and contributions in the next section. 

2. Problem Statement and Main Contributions 

We consider the design of low dimensional (compressive) measurement 
matrices, with a pre-specified number of measurements, for detecting sparse 
signals in additive white Gaussian noise. More specifically, we consider the 
following binary hypothesis test: 

"H :x = n, 

Hi :x = s + n, { } 

where x is an (N x 1) vector that describes the state of a physical phe- 
nomenon. Under the null hypothesis Hq, x is a white Gaussian noise vector 
with covariance matrix B[nn ] = (<7^/iV)I. Under the alternative hypoth- 
esis Hi, x = s + n consists of a deterministic signal s distorted by additive 
white Gaussian noise n. 



We assume s is /c-sparse in a known basis \I/. That is, to say, s is 
composed as 

s = *0, (2) 

where \I/ £ M is a known matrix, whose columns form an orthonormal 

basis for M. N , and G M. N is a fc-sparse vector, i.e., it has between 1 to 
k <C iV nonzero entries. We may refer to s as simply fc-sparse for brevity. 

We wish to decide between the two hypotheses based on a given number 
m < N of linear measurements y = 3>x from x, where $ £ i mxW is 
a compressive measurement matrix that we will design. The observation 
vector y = <l?x belongs to one of the following hypothesized models: 

^o:y = *n-AA(0,(a2/iV)**^), 

Hi : y = *(s + n) ~ jV(*s, (c^/iV)**^), l > 

where the superscript H is the Hermitian transpose. To avoid coloring the 
noise vector n, we constraint the compressive measurement matrix 3> to be 
right orthogonal, that is we force $$ H = I. 

Rather than limiting ourselves to a particular detector, we look at the 
general problem of designing compressive measurements to maximize the 
measurement signal-to- noise ratio (SNR), under "Hi, which is given by 

SNR= (s H ® H $s)/(al/N). (4) 

This is motivated by the fact that for the class of linear log-likelihood ratio 
detectors, where the log-likelihood ratio is a linear function of the data, the 
detection performance is improved by increasing SNR. In particular, for a 
Neyman-Pearson detector, with false alarm rate 7, the probability of detec- 
tion Pd = Q{Q~ 1 {'~i) — vSNR) is monotonically increasing in SNR, where 
Q{-) is the Q-function. In addition, maximizing SNR leads to maximum 
detection probability, at a pre-specified false alarm rate, when an energy de- 
tector is used. Without loss of generality, throughout the paper we assume 
that a\ = 1 and ||s|| 2 = \\0\\ 2 = 1, and so we design $ to maximize the 
measured signal energy ||$s|| 2 . 

In solving the problem, one approach is to assume a value for the sparsity 
level k and design the measurement matrix $ based on this assumption. 
This approach, however, runs the risk that the true sparsity level might be 
different. An alternative approach is not to assume any specific sparsity 
level. Instead, when designing the measurement matrix 3>, we prioritize the 
level of importance of different values of sparsity k. In other words, we first 
find a set of solutions that are optimal for a fci-sparse signal. Then, within 
this set, we find a subset of solutions that are also optimal for /c2-sparse 



signals. We follow this procedure until we find a subset that contains a 
family of optimal solutions for sparsity levels k\ , hi-, k^, ■ ■ ■ . This approach 
is known as a lexicographic optimization method (see, e.g., |21j-j23j). 
Replacing ^ in Q yields 

||$*0|| 2 

The basis matrix \I/ is known, but the /c-sparse representation vector 9 is 
unknown. That is, the exact number of the nonzero entries in 6, their 
locations, and their values are unknown. The measurement design naturally 
depends on one's assumptions about the unknown vector 6. We consider two 
different design problems, namely a worst-case SNR design and an average 
SNR design, as explained below. 

Worst-case SNR design. In the first case, we assume the vector 6 is 
deterministic but unknown. Then, among all possible deterministic A:-sparse 
vectors 6, we consider the vector that minimizes the SNR and design the 
matrix <1? that maximizes this minimum SNR. Of course, when minimizing 
the SNR with respect to 0, we have to find the minimum SNR with respect 
to locations and values of the nonzero entries in the vector 9. To combine 
this with the lexicographic approach, we design the matrix <1? to maximize 
the worst-case detection SNR, where the worst-case is taken over all subsets 
of size ki of elements of 6, where ki is the sparsity level considered at the 
ith level of lexicographic optimization. This is a design for robustness with 
respect to the worst sparse signal that can be produced in the basis \I/. The 
reader is referred to Section [3] for a complete statement of the problem. 

We show (see Section H| that the worst-case detection SNR is maximized 
when the columns of the product 3>Vl/ between the compressive measurement 
matrix $ and the sparsity basis \l/ form a uniform tight frame. A uniform 
tight frame is a frame system in which the frame operator is a scalar multiple 
of the identity operator and every frame element has the same norm (see, 
e.g., [21] )• We also show that when the signal is 2-sparse, the optimal frame 
is a Grassmannian line packing (see, e.g., |25jb For the case where the 
sparsity level of the signal is greater than two, we provide a lower bound on 
the worst-case performance. If the number m of measurements allowed is 
greater than or equal to yN, then the Grassmannian line packing frame will 
be an equiangular uniform tight frame (see, e.g., |26j-|33j and the maximal 
worst-case SNR can be expressed in terms of the Welch bound. Numerical 
examples presented in Section[7|show that Grassmannian line packing frames 
provide better worst-case performance than matrices with i.i.d. Gaussian 
entries, which are typically used in sparse signal reconstruction. 



Average SNR design. In the second case, we assume that the locations 
of nonzero entries of are random but their values are deterministic and 
unknown. We find the matrix 3? that maximizes the expected value of the 
minimum SNR. The expectation is taken with respect to a random index 
set with uniform distribution over the set of all possible subsets of size ki 
of the index set {1,2,..., N} of elements of 0. The minimum SNR, whose 
expected value we wish to maximize, is calculated with respect to the values 
of the entries of the vector for each realization of the random index set. 
The reader is referred to Section [5] for a complete statement of the problem. 

We show (see Section rc3J) that for 1-sparse signals, any right orthogonal 
measurement matrix <&, i.e., any tight frame, is optimal for maximizing the 
average minimum SNR. For signals with sparsity levels higher than one, 
we constrain ourselves to the class of uniform tight frames and show that 
optimal measurement matrix is a uniform tight frame that has minimal sum- 
coherence, as described in Section [6j However, to the best of our knowledge 
constructing such frames remains an open problem in frame theory. There- 
fore, we limit ourselves to providing performance bounds in the average-case 
problem. 

3. The Worst-case Problem Statement 

Since all sparse signals share the fact that they have at least one nonzero 
entry, it seems natural to first find an optimal measurement matrix for 1- 
sparse signals. Next, among the set of optimal solutions for this case, we find 
matrices that are optimal for 2-sparse signals. This procedure is continued 
for signals with higher sparsity levels. This is a lexicographic optimization 
approach to maximizing the worst-case SNR. 

Consider the kth. step of the lexicographic approach. In this step, the 
vector 6 has up to k nonzero entries. We do not impose any prior constraints 
on the locations and the values of the nonzero entries of 6. As mentioned 
earlier, we assume that ||s|| 2 = \\0\\ 2 = 1 and a 2 = 1. We wish to maximize 
the minimum (worst-case) SNR, produced by assigning the worst possible 
locations and values to the nonzero entries of the /c-sparse vector 6. Referring 
to Q, this is a worst-case design for maximizing the signal energy s^3>^<l>s 
inside the subspace (<& ) spanned by the columns of <& , since $^$ is the 
orthogonal projection operator onto ($> H )- 

To define the fcth step of the optimization procedure more precisely, we 
need some additional notation. Let Ao be the set containing all (mxN) right 
orthogonal matrices 3>. Then, we recursively define the set Ak, k = 1, 2, ... , 
as the set of solutions to the following optimization problem: 



max min 
c e 

s.t. 


I|C0|| 2 , 
C G B fc _i, 




ll»ll = l, 




is /c-sparse 



maxmin ||<&s|| , 

# s 

s.t. * G A k -i, (5) 

ll s ll = !> 

s is /c-sparse. 

In our lexicographic formulation, the optimization problem for the /cth prob- 
lem (J5]) involves a worst-case objective restricted to the set of solutions Ak-i 
from the (k — l)th problem. So, Ak C «4fe-i C • • • C ^4o- 

Before we present a complete solution to these problems, we first simplify 
them in three steps. First, since the matrix \1/ is known, the matrix 3> 
can be written as $ = C\& , where C is an (m x N) matrix. Then, 
$* = C* H * = C, and also **^ = C* H *C H = CC H = I. Using (g]), 
the max-min problems (pi) become 



(6) 



where Bo = Aq, and similar to the sets Ak, the sets B k (& = 1,2,...) are 
recursively defined to contain all the optimal solutions of ([6]). It is easy to 
see that B k = {C : C*^ G Ak}- 

Let n = {l,2,...,N} and define n k to be n k = {E C 9. : \E\ = k}. For 
any T G O^, let 0y be the subvector of size (fcx 1) that contains all the 
components of 9 corresponding to indices in T. Similarly, given a matrix C, 
let Ct be the (m x k) submatrix consisting of all columns of C whose indices 
are in T. Note that the vector Ot may have zero entries. Indeed, for cases 
where the /c-sparse vector has fewer than k, e.g., I < k, nonzero entries, 
the (k x 1) vector Ot has k — I zero entries. This is important because our 
definition for T and Ot is slightly different than the common definitions used 
in the compressed sensing literature, where T and Ot only contain indices 
and values related to the nonzero entries of the vector 0, often called the 
support of T. We refer to a member T of £l k as a "/c-platform" . Thus, a 
/c-platform T includes, but is not limited to, the support of the sparse vector 
0. 

Given T G Q k , the product CO can be replaced by CtOt instead. Now, 
to consider the worst-case scenario for the SNR, as well as considering the 
worst Ot that minimizes ||Ct0t|| 2 ) we also have to consider the worst T G 



Ofe. Thus, the max-min problem becomes 

max min min IICt^tII 2 , 
c T e T 

s.t. C G H fe _i, (7) 

||0 r || =l,Tefi fe . 

The solution to ([7]) is the most robust design with respect to the locations 
and values of the nonzero entries of the parameter vector 0. 
The solution to the minimization subproblem 

min ||Ct#t|| 2 , 

s.t. \\e T \\ = 1, 

is well known; see, e.g., [53] . The optimal objective function is A m i n (C^Or), 
the smallest eigenvalue of the matrix C^Ct- Therefore, the max- min- min 
problem ([7]) simplifies to 



(Pfc) 



max min 
C T 

S.t. 


C G B fc _i, 




r g fi fe . 



(8) 



At each step k, the optimal compressive measurement matrix, denoted by 
<!>*, is determined from the optimizer C* of ([8]) as <&* = C**& . 
Next, we describe how to solve the max-min problem (Pj.) in ([8]). 

4. Solution to the Worst-case Problem 

Let Cj be the ith column of the matrix C. We first find the solution set 
A\ for problem (Pi). Then, we find a subset A2 C >4i as the solution for 
(P2). We continue this procedure for general sparsity level k. 

4-1. Sparsity Level k = 1 

If k = 1, then any T such that \T\ = 1 can be written as T = {i} 
with i £ fi, and Ct = Cj consists of only the ith column of C. Therefore, 
C^Ct = c^c, = ||cj|| 2 , and the max-min problem becomes 

maxmin ||cj|| 2 , 

s.t. l CeB , (9) 



Theorem 1. The optimal value of the objective function of the max-min 
problem |M) is m/N. A necessary and sufficient condition for a matrix C* 
to be in the solution set B\ is that the columns {c*}fL 1 of C form a uniform 
tight frame with norm values equal to ^Jm/N . 

Proof. We first prove the claim about the optimal value. Assume false, i.e., 
assume there exists an optimal matrix C* G B\ for which the value of the 
cost function is either less than or greater than m/N. Suppose the former is 
true. Let Ci be an (m x N) matrix, satisfying CiCf^ = I, whose columns 
have equal norm ^Jm/N . Then, the value of the objective function in ([9]) 
for C = Ci is m/N . This means that our proposed matrix Ci achieves a 
higher SNR than C* which is a contradiction. Now, assume the latter is 
correct, that is the value of the objective function for C* is greater than 
m/N . This means min lie* II 2 = ||c*|| 2 > m/N . Knowing this, we write 

N N 

tr (C*C* H ) = tr (C*^C*) = J^ ||c* || 2 > J^ m/N 



N N 

m. 

i=l i=l 



However, from the constraint in (19J) we know that C*C = I, and tr(C*C ) 
m. This is also a contradiction. Thus, the assumption is false and the opti- 
mal value for the objective function of (I9| is m/N . 

We now prove the claim about the optimizer C*. From the preceding 
part of the proof, it is easy to see that all columns of C* must have equal 
norm sJm/N. If not, since none of them can be less than y/m/N, then the 
sum of all column norms will be greater than m, which is a contradiction. 
Moreover, we write 

N 
C * C *H = J2 C * C *H = l ( 10 ) 



Multiplying both sides of (10) by an arbitrary (m x 1) vector x from the 
right side and x from the left side, we get Y^i=i ll c i x l| 2 = ll x l| 2 - This 
equation represents a tight frame with frame elements {c*} and frame bound 
1. In other words, it represents a Parseval frame. Since the frame elements 
have equal norms, the frame is also uniform. Therefore, for a matrix C* to 
be in B\, the columns of C* must form a uniform tight frame. □ 



Remark 1: The reader is referred to [24|, [29], [35], [36], and the refer- 
ences therein, for examples of constructions of uniform tight frames. 



4-2. Sparsity Level k = 2 

The next step is to solve (P2)- Since our solution for this case should 
lie among the family of optimal solutions for k = 1, results concluded in 
the previous part should also be taken into account, i.e., the columns of 
the optimal matrix C* must form a uniform tight frame, where the frame 
elements c* have norm y/m/N . 

Given T G CI2, the matrix Ct consists of two columns, e.g., Cj and Cj. 
So, the matrix C^Ct in the max-min problem (I8J) is a (2 x 2) matrix: 



Kyrp (_, rp 



\Ci,Cj) \Cj,Cj) 



From the k = 1 case, we have ||cj|| 

(m/N) 



m/N . Therefore, 



C ji Ct 



cos a,; 



cos a 
1 



u 



where a^- is the angle between vectors Cj and c,-. The minimum eigenvalue 
of this matrix is 

A min (C^C T ) = (m/N)(l - I cosoijl). 
Given any matrix C S B±, define coherence fie as 



^c = max j| — J 

Ci,Cj: columns of C Cj Cj 



Also, let n* be 

The following theorem holds. 



a = mm un- 

ceBi 



(11) 

(12) 
(13) 



Theorem 2. The optimal value of the objective function of the max-min 
problem (P2) is (m/N)(l — /x*). A matrix C* is in B2 if and only if the 
columns of C* form a uniform tight frame with norm values y/m/N and 
fi c * =/x*. 

Proof. Since our solution must be chosen from the family of uniform tight 
frames with frame elements of equal norm y/m/N, the objective function 
of (P2) is only a function of the angle aij. Using (11) and (12), it is easy 

(m/N)(l - n c ). Using (pi), we 



to see that the minimum A r 



(C£C T ) 



is 



conclude that the largest possible value of the objective function of (P2) is 
(m/N)(l-ff). D 
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Remark 2: Methods for constructing uniform tight frames with frame 
elements that have a coherence fj,* is equivalent to optimal Grassmannian 
packings of one-dimensional subspaces, or Grassmannian line packings (see, 
e.g., [25]-[33]). We will say more about this point later in the paper. 

Remark 3: In the case where k = 2, the matrix C^Ct (where C G £>i), 
for any choice of T £ £1%, is a (2 x 2) matrix with minimum and maximum 
eigenvalues equal to (m/iV) (1±| cosajj|). Therefore, the matrix C^Cy with 
eigenvalues equal to {m/N){l ± fie) has the smallest minimum eigenvalue 
and the largest maximum eigenvalue among eigenvalues of all matrices of the 
form dpCr (for a fixed C and a varying T). Moreover, among all C £ Bi, 
when comparing the resulting submatrices C^Ct for T G f^, the matrix C* 
with coherence [i* has the largest minimum eigenvalue (m/N)(l — fj>*) and 
the smallest maximum eigenvalue (m/N)(l + /i*). This means that given 
any vector s G R 2 and T 6 Q2, the following inequalities hold: 

(l-^)||s|| 2 < ||C^s|| 2 <(l + //)||s|| 2 . (14) 

Recall the definition of Restricted Isometry Property (RIP) (see, e.g., 
[6]): Let A be a (p x q) matrix and let I < q be an integer. Suppose Si > 
is the smallest constant such that, for every (p x /) submatrix A; of A and 
every (Ixl) vector s, 

(1 — ^)||s|| 2 < N^V^sll 2 < (1 -P ^z)||s|| 2 . 

Then, the matrix A is said to satisfy the Z-restricted isometry property 
(/-RIP) with the restricted isometry constant (RIC) 5[. 



By comparing the 2-RIP definition with ( 14 ) , we can conclude that the 
optimal matrix C* not only satisfies the 2-RIP with RIC fj,* , but also among 
all matrices that satisfy 2-RIP and have uniform column norms equal to 
^m/N ', it provides the best RIC. Thus, our solution for optimizing the 
worst-case SNR for 2-sparse signals is also the ideal matrix for recovering 
2-sparse signals based on methods that rely on the RIP condition for their 
performance guarantees. 

4-3. Sparsity Level k > 2 

We now consider the case where k > 2. In this case, T £ 0,^ can be 
written as T = {«i, «2, • • • , ik} C CI. From the previous results, we know 
that an optimal matrix C* £ Bk must already satisfy two properties, in 
addition to C*C*^ = I: 

• Columns of C* must build a uniform tight frame with equal norm 
y/m/N (to be in the set Si), 

11 



• The coherence /ic* should be equal to //* (to be in the set B2). 

Taking the above properties into account for C*, the matrix C^C^ will 
be a (k x fc) symmetric matrix that can be written as CjP CJ. = (m/iV)[I + 
Ay] where At is 




cos a 

cos a 



nn 



cos a 




«1«2 



COSQ 
COSQ 



*2*fc 



U*fc 



COSQ 



I2'k 







(15) 



where ih / i/ G T for the entry cosa*. in the (ih,if)th location. Then, 

A min (C^C^) = (m/N)(l + Andn(Ar)). (16) 

So, the problem simplifies to 



(Pfc) 



maxmin A m i n (Ar), 

C T 

s.t. C e B fe _i, 
Tefi t . 



(17) 



Solving the above problem is not trivial. It is worth mentioning that, as we 
will discuss later, the family of frames lying in the set B2 are known to be 
Grassmannian line packings. Building such frames is known to be very hard 
and in fact, for a lot of values of m and N, no solution has been found so 
far (see, e.g., [25J ). This means that building solutions for problems (P&) 
is even a harder task. Nevertheless, we provide bounds on the value of the 
optimal objective function. 



Given T G Sl k , let 5* t be 



d i h if=V -\cosa ihif \, 
Also, define A* in the following way: 



i h / if G T. 



(18) 



A* = min V 5* ihit . 



i h ^i f £T 



The following theorem holds. 



Theorem 3. The optimal value of the objective function of the max-min 
problem (Pk) for k > 2 lies between (m/N)(l — ( 2 )m* + A*) and (m/N)(l — 



12 



Proof. Let Xjj and y~ij be two (k x 1) vectors such that Xjj contains values 
(l/\/2) and (— l/\/2) an d yjj contains values (l/\/2) and (l/\/2) in the ith 
and jth locations (i 7^ j) and zeros elsewhere. Then, by using Rayleigh's 
inequality, i.e., 

x^A T x 



A 



uunV-A/r) < 



X H X 



for the matrix At defined above and the family of vectors {x,j} and {yij} 
defined by i and j (chosen from the set {1, 2, . . . , k}), we conclude that 
A min (A T ) < -| cosa| hi/ |, ih / if G T. Thus, 



min A m i n (A-r) < min ( 

T 6 fi fe 



cos a 



ihtfi 



"M 



(19) 



Given T G fife, the matrix Ay can be written as summation of ( 2 ) 
matrices Fi h i f (ih 7^ if G T) where each matrix F^^ has the entry cosa* ^ 
in the (i/j,z^)th and (if,ih)ih locations and zeros elsewhere. Using matrix 
properties (see, e.g., [38])) we can write 



A min (A T ) > Y^ X mm(F lhlf )= Yl -\ cosa hf\ 
ren k T<=n k 



ren fc 



h*/ 



ren fc 



Therefore, 



mm A min (A T ) > - 1 '" )// + A*. 



(20) 



Using (16), (19), and (20) we get 



(m/iV)(l - ix*) > min A min (C^C^) 



Ten fc 
> (m/N)(l 



H* + A*) 



This completes the proof. 



(21) 

□ 



4-4- Equiangular Uniform Tight Frames and Grassmannian Packings 

The inequality (21) in Theorem ([3]) suggests that if all angles between 
column pairs are equal, then the optimal value of the objective function of 
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(Pfc) for k > 2 will reach its upper bound. In this case, the columns of 
C* G £>fc in fact form an equiangular uniform tight frame. 

Equiangular uniform tight frames are Grassmannian packings, where a 
collection of N one-dimensional subspaces are packed in W 1 such that the 
chordal distance between each pair of subspaces is the same (see, e.g., [25] . 
|27j . and [28J). Each one-dimensional subspace is the span of one of the 
frame element vectors Cj. The chordal distance between the iih. subspace 
(cj) and the jth subspace (cj) is given by 

d c (i,j) = ysin 2 a 4j , (22) 

where a%j is the angle between c% and Cj. When all the ctij, i 7^ j, are equal 
and the frame is tight, the chordal distances between all pairs of subspaces 
become equal, i.e., d c (i,j) = d c for all i 7^ j, and they take their maximum 
value. This maximum value is the simplex bound given by 

d c = y/(N(m-l))/(m(N-l)). (23) 

Alternatively, the largest absolute value of the cosine of the angle between 
any two frame elements is bounded as 



max cos a% 



¥j 



1.) 



> y/(N-m)/(m(N-l)). 



The derivation of this lower bound is originally due to Welch [39]. The 
Welch bound, or alternatively the simplex bound, are reached if and only if 
the vectors {ci}^L 1 form an equiangular uniform tight frame. This is possible 
only for some values of m and N. It is shown in [32] that this is possible 
only when 1 < m < N — 1 and 

N < min{m(m + l)/2, (N -m)(N -m+ l)/2} (24) 

for frames with real elements, and 

iV< min{m 2 ,(iV-m) 2 } (25) 

for frames with complex elements. If the above conditions hold, then the 
optimal solution for (Pj.) for k > 2 is a matrix C* such that its columns 
form an equiangular uniform tight frame with frame elements of equal norm 
^JmjN and angle a defined as 




« = arcsin J — - — — . (26) 
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The optimal value of the objective function of (P&) in this case is (m/N)(l — 
li*), where [m* = \ cosa\ = y/(N - m)/(m(N - 1)). 

In other cases where A^ and m do not satisfy the condition (24) or (25), 
the following inequality provides a tighter bound than the simplex bound 
for ji* for some values of N and m (see |40| ) : 



ft* > cos 



f( m -i)T(^)\ 1/(m ' iy 



n^ rm 



Applying the above inequalities to (21), we conclude that by using a Grass 



mannian line packing where the k largest angles among angles between col- 
umn pairs of the matrix C* are as close as possible to the angle a related 
to fi* , the value of the SNR is guaranteed to be higher than the computed 
lower bound. This is, however, a very difficult problem since even finding 
Grassmannian line packings for different values of A~ and m is still an open 
problem. The reader is referred to [25j and |28j for more details. 

We have thus considered a worst-case design criterion in which we as- 
sume nothing about the vector 6, and our design is robust against arbitrary 
possibilities of this unknown. 

5. The Average-case Problem Statement 

In the worst-case problem, an optimal /c-platform T for problem (P^) is 
a member of ilk that minimizes ||Ct#t|| 2 - In this section, instead of finding 
the worst-case T, we consider an average-case problem with a random T. 
Let Tfc to be a random variable taking values in 0&, uniformly distributed 
over Ofc. In other words, if we let Pk(t) be the probability that T^ = t where 
t €.ttk, then 

Pk(t) = (A , vt e n k . 

Our goal is to find a measurement matrix $ that maximizes the expected 
value of the minimum SNR, where the expectation is with respect to the 
random /c-platform T)%, and the minimum is with respect to the entries of 
the vector 6 on T^. Taking into account the simplifying steps used earlier 
for the worst-case problem in Section [3] and also adopting the lexicographic 
approach, the problem of maximizing the average SNR can then be formu- 
lated in the following way: Let A/"o be the set containing all (m x N) right 
orthogonal matrices. Then for k = 1, 2, . . . , recursively define the set A4 as 
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the solution set to the following optimization problem: 

maxET, min II Or, #fc|| 2 , 
c k Ok 

s.t. C € A4-1, ( 27 ) 

\\(tk II = 1) 

where Ey fc is the expectation with respect to T^. As before, the (m x A;) 
matrix Cx k ar e the columns of C whose indices are in T^. The above can 
be simplified to the following: 

{max Etv A m ; n (C^ Cr, ), 
C ^ ^ *i ^' (28) 

s.t. CeJVfc-i- 

6. Solution to the Average-case Problem 

To solve the lexicographic problems (F&), we follow the same method we 
used earlier for the worst-case problem, i.e., we begin by solving problem 
(Fi). Then, from the solution set Mi, we find optimal solutions for the 
problem (F2), and so on. 

6.1. Sparsity Level k = 1 

Assume that the signal s is 1-sparse. So, there are ( 1 ) = N different 
possibilities to build the matrix Cj^ from the matrix C. The expectation 
in problem (Fi) can be written as: 

N 

E Tl A min (C£c Tl ) = Y, Pi(t)A min (Cf C 4 ) = 5>i(W)|| Cj || 2 = ^. (29) 
tefii «=i 

The following result holds. 

Theorem 4. The optimal value of the objective function of problem (Fi) is 
m/N . This value is obtained by using any right orthogonal matrix C £ Mq, 
i.e., any tight frame. 

Proof. The first part is already proved. The proof for optimality is very 
similar to the proof given in Theorem [TJ Thus, Mi = Mq. □ 

Theorem [4] shows that unlike the worst-case problem, any tight frame is 
an optimal solution for the problem (Fi). 

Next, we study the case where the signal s is 2-sparse. 
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6.2. Sparsity Level k = 2 

For problem (F2), the expected value term ET 2 A m i n (C^ 2 CT 2 ) is equal to 

N j-1 

£ p 2 (i)A min (Cf C t ) = ^^ P2 ({i,i})A min (C^. } C {M} ). 
ten 2 j=2 i=i 

Now, since p 2 (*) = l/(^) = 2/(N(N - l)),Vi G fi 2 , we can go further and 
write Er 2 A m ; n (C^ 2 Cr 2 ) as 

- ^^> ,.-,// ^ ^ (30) 



N(N-l) S 1^ A min(C^ i} C fe} ). 
V ; 7=2 i=l 



Solving problem (F 2 ) with this objective function is not trivial in general. 
In fact, claiming anything about solutions of the family of problems (F&), 
k = 2,3,..., is hard. However, if we constrain ourselves to the class of 
uniform tight frames, which also arise in solving the worst-case problem, we 
can establish necessary and sufficient conditions for optimality. Nonetheless, 
these conditions are different from those for the worst-case problem and as 
we will show next the optimal solution here is a uniform tight frame for 
which a cumulative measure of coherence is minimal. 

Let Mi be defined as Mi = {C : C G A/i, ||c*|| = y/m/N, \/i G fi}. 
Also, for k = 2,3,..., recursively define the set Mk as the solution set to 
the following optimization problem: 

, f max E T A min (Cy CtJ, , N 

F c * V T * (31) 

[ s.t. C G M k -i- 

We will concentrate on solving the above problems instead of the family of 
problems (F&), k = 2, 3, . . .. For k = 2, we have the following result. 

Theorem 5. The matrix C is in M2 if and only if the frame sum-coherence 
Ej=2 Ei=i K c «) c i)l is minimized. 

Proof. For k = 2, the value of A m ; n (C^C t ) for t = {i,j} G S7 2 is equal to 
A min (C^. } C {M . } ) = (l/2)(||c,|| 2 + ||c,|| 2 - f(i,j)), 



where f(i,j) is defined as f(i,j) = v^ll *!! 2 — ||cj|| 2 ) 2 + 4(cj, Cj) 2 . Now, if 
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we replace this in ( 30 ) , we get 



N j-l 

2 , n_ i|2 



1 Wllc-ll-^-lle- 



K j=2 i=l 

N 



i=l 

N j-l , N j-l 



= ]v^((^- 1 )Eni 2 -EE/(^) 

= (N-i)m Lfr f(i 7 - ) = ™ ' rr f(j 7 -) 

iV(iV-l) N(N-1) 4^t^ ^ N(N-\)4^4^ 

7 — ^ & — X 7 — ^ o — J- 

Since C 6 .Mi, then using the fact that ||cj|| = y/m/N, Vz G 0, we can go 
one step further and write the above objective function as 

o N j-l 

m 2 



N AT(iv--i)EEi^' c ^i- 

V ; 3=2 i=l 

Therefore, solving problem (F 2 ) becomes equivalent to solving the following 
optimization problem: 



mi n E 7 = 2 £i=i |<Ci, c^->|, 



c ^ 

s.t. C£jMi 



(32) 



D 



Theorem p^ shows that for problem (F 2 ), angles between column pairs of 
the uniform tight frame C should be designed in a different way than for the 
worst-case problem. Several articles (though not many) discuss such frames. 
In [5T] , the authors introduce a similar concept where instead of finding the 
minimum of the above summation, they are looking for the maximum, and 
call it the "cluster coherence" of the frame. In [29], where the authors use 
frames in coding theory applications, it is proved that the solution to one 



of the problems discussed in the paper is found by solving (32). However, 



to the best of our knowledge, finding such a frame system is still an open 



problem — there is no known general solution for problem ( 32 ) . We call the 



value of the optimal objective function of (32) the minimum sum-coherence. 
The following lemma provides bounds for the objective function of this op- 
timization problem. 
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Lemma 1. For a uniform tight frame C with column norms equal to yfm/N, 
the following inequalities hold: 

N j-\ 

ab\(N/m - 1) - 2(iV - 1)/£| < ^ ^ K*, Cj )\ < ab(N - l)/£, 

j=2 t=l 

where 

(m/N) 2 \ (N{N-2) 



( (m/Nf \ A 

\l-2(m/N))> ' "V 



Proof. See Appendix A. 

6.3. Sparsity Level k > 2 

Similar to the worst-case problem, solving problems (F fc ) for k > 2 is not 
only a hard task but also it is not known how to construct frames with the 
required properties in practice. This is because the solution sets for these 
problems all lie in M.2 and the problem (F 2 ) is still an open problem. The 
following lemma provides a lower bound for the optimal objective function 
of problem (F fe ). 

Lemma 2. The optimal value of the objective function for problem (F k ) is 
bounded below by (m/N)(l — {k{k — l)/2)/j,c)- 

Proof. See Appendix B. 

7. Simulation Results 

As mentioned earlier, constructing uniform tight frames with coherence 
(j,* is an open problem for arbitrary (m, N) pairs. However, examples of such 
frames are available for modest values of m and N, mostly for 1 < m < 16 
and 1 < N < 50 (see [42J). To be more precise, the examples in |42j are the 
best uniform tight frames (in terms of coherence) that the site publisher is 
aware of. In some cases, these frames in fact have coherence /j,* . In other 
cases, their coherence is larger than //*. For the minimum sum-coherence 
problem, the examples are even more scarce, and in fact we are not aware 
of any examples for (m, M) dimensions large enough to be of interest to our 
study. Therefore, we limit our numerical study to the worst-case problem, 
where we evaluate the performance of several uniform tight frames from |42j . 

In all simulations, we assume a 2 = 1 and ||0t|| = 1- We present plots of 
the worst-case SNR/A r , where the worst-case SNR is given by 

SNR = minmin||** T T || 2 /(^,/A r ) = min X min {C* T H C* T ) , 
T O^p T 
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by fixing two of the three variables m, N, and k and changing the third one. 
We compare the performance of our robust (worst-case) design C* with that 
of a matrix R with i.i.d. Gaussian jV(0, (1/m)) entries, which is typically 
used for signal recovery. To satisfy the constraint in problem ([8]) , we make R 
to be right orthogonal. The value of the objective function in (I8j is averaged 
over 100 realizations of the matrix R. 

Figure [It a) shows the worst-case SNR performance for a case where the 
signal dimension is N = 50 and the measurement budget m is varied from 



10 to 40. In this case, the condition (24) is satisfied and the columns of 
the optimal matrix C* form an equiangular uniform tight frame. We can 
therefore derive an exact expression for the optimal objective function value 
based on the Welch bound. For k = 1, this value is equal to m/N, and for 
k > 2, it is equal to (m/N) (1 - //*) where // = yj [N - m) / (m{N - 1)). 



We also consider cases where the condition ( 24 ) is not satisfied, due to a 
relatively small measurement budget. Here we use Grassmannian line pack- 
ings to form measurement matrices. For (JV, m) pairs that Grassmannian 
line packings are not known, we use the best available packings reported in 
|42j for those dimensions. Figures mb)-(f) show the performance of such so- 
lutions versus the random matrix R for different case. In each case, we have 
fixed two of the variables N, m, and k and have varied the third one. The 
values of the objective functions in all these plots are in dB. In all scenarios, 
the wort-case SNR performance corresponding to the the optimal design C* 
is better than the average taken over 100 realization the random matrix R. 

Note that our simulations are only for cases where m, k and N are not 
very large. As mentioned above, one of the reasons is that the available 
uniform tight frames in [32] are mostly for cases where 1 < m < 16 and 
1 < N < 50. Also, for values of N bigger than 25 and k bigger than 5, 
finding the smallest minimum eigenvalue of all C^ C^ for different values 
of T is computationally intractable. 

It is important to realize that for most values of m and N, the uniform 
tight frames used in our simulations have a coherence \i that is bigger than 
[i*. In other words, for most values of m and N, we are actually comparing 
the performance of a suboptimal solution matrix instead of the optimal 
solution with the performance of the random matrix R and interestingly, 
the suboptimal solution still has a better performance than the random 
matrix R in most, but not all, cases. For example, we notice that in Figure 
1(f), the gap between two curves decreases as N increases. This does not 
contradict with our theoretical results, as the plots in Figure [l] do not show 
the performance of the optimal solution for most values of m and N after 
all. Rather they show the performance of the best available uniform tight 
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Figure 1: Performance comparison between matrices C* and R: (a) equiangular uniform 
tight frames, (b) m — 8, N = 50, and k is varied, (c) k — 2, N — 40, and m is varied, (d) 
k — 3, N — 40, and m is varied (e) m — 10, k — 3, and iV is varied, (f) m — 10, k = 4, 
and N is varies. 
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frame example for the corresponding (rn, N) values. 

8. Conclusions 

In this paper, we have considered the design of low-dimensional (com- 
pressive) measurement matrices, for a given number of measurements, for 
maximizing the worst-case SNR and the average minimum SNR. We have 
shown an interesting connection between maximizing the two SNR criteria 
for detection and certain classes of frames. In the worst-case SNR problem, 
we have shown that the optimal measurement matrix is a Grassmannian line 
packing for most — and a uniform tight frame for all — sparse signals. In the 
average SNR problem, we have looked for the solution among the class of 
uniform tight frames and have shown that the optimal measurement matrix 
is a uniform tight frame that has minimum sum-coherence. Our solutions for 
both problems provide lower bounds for the performance of the detectors. 

Appendix 

A. Proof of Lemma 1. 

Multiply both sides of QC H = I from the left by C H and from the right 
side by C to get 

(C H C) 2 = C^C. (33) 

The matrix C C = I is an (N x N) Hermitian matrix, with (i,j)th element 
(m/N) cos ctij and diagonal elements m/N. Using these values, it is easy 
to see that the matrix (C C) 2 is also a Hermitian matrix with the entry 
(m/N) 2 (^2 i=1 cos 2 aji) on the jth. diagonal location and the entry 

N 

(m /N) 2 (2 cos a>ij + \, cos a il cos a lj) 
located in the ith row and the jth column. By comparing the diagonal 



entries on each side of equation (33), we will get the following family of 
equations: 
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If we sum up all the above equations, after simplifying, we getjj 

N 2 



JV 
y COS" n 



./' 



I'll 



(34) 



If we compare the off-diagonal entries of matrices on each side of equa- 



tion (|33[) , then for i, j = 1, . . . ,N and i / j, we get 

V ( ^ 

. I 2 cos aij + >^ cos a# cos a^ 



my 






COS «jj , 



which simplifies to 



cos a 



'j 



(m/N) 
1 - 2{m/N) 



j y^ cos an cos aij 



Using the triangle inequality, we write 

N j-l N j-l 

j=2 j=l j=2 i=l 



COS Q ? - 



> 






iV j-l 

EE- 

j=2 i=l 



AT j-l N 

y y y cos an cos a;j 

1=2 j=l 1=1, 

We replace cos an cos a^- with (l/2)(cos 2 an + cos 2 a/j — (cos an — cos aij) 2 ). 
The term cos 2 ckj; is repeated 2(iV — 2) times in the above summation; 

• Once i and I are fixed, there are N — 2 choices left for j to choose the 
angle a^ in the product term cos an cos a^ . 



J The relation (34 1 is the well-known frame potential condition (see [43] ) 

JV 

FP= Y. I<ci,cj)| 2 =m 



»ij'=i 



for tight frames, after it has been simplified by enforcing the equal norm assumption. 
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There are also N — 2 times that the term cos aji is repeated, which is 
equal to cos ay . 



Therefore, 



N j-1 N N j-1 

y > , / cos a ii + cos Qry = 2(iV — 2) Y^ V. cos a « 

<—, ^ <—, ^ _^ 



7=2 i=l 4=1, 



2(7V - 2; 



(AT 2 / m ) - AT 



The right hand side of the above inequality simplifies to 



(a/2) 



W j-1 AT 

N(N - 2)(N/m- 1)-$I$Z ^(cosa^ -cos ay) 



1=2 i=i f=i, 



It is easy to show that | cos an — cos ay | < 2/jc f° r an Y * 7^ i 7^ I = 1> • • ■ > -W- 
So, 

Af j-1 Af Af j-1 AT 



7=2 j=l i=l, 



7=2 i=l (=1, 



Similarly, for a fixed i and j, there are N — 2 possibilities for Z. Also, there 
are ( 2 ) ways to choose i and j from AT options. Therefore, the lower bound 
will be larger than 

(a/2) \N(N - 2)(N/m - 1) - 2N(N - 1)(N - 2)/j, 2 c \ 
= ab\(N/m - 1) - 2(AT - l)/j, 2 c \. 

This is the claimed lower bound. 



21 



To find the upper bound, we write 



N j-l 

EEK« 

j=2 i=l 



m 

N 



N j-l 
2 _, / | COS ' \ 
j=2 i=l 



u 



«EE 

J=2 4=1 



A? 



7 COS «j; COS Cfyj 






AT j-l N 

< a y^ 2, /, I cos a u cos a lj I 

1=2 j=l 1=1, 

N j-l N 

j=2 i=l 1=1, 

= ab(N -l)fi 2 c . 



□ 



5. Proof of Lemma 2. 

Similar to the 2-sparse signals case in Section |4j we can write the objec- 
tive function of problem (F fc ) in the following way: 



ET fe A m i n (C Tft CT fe ) = 2_j Pk(t) A m in(C t C t ) 

teOfc 



N 



-i 



/ A m i n (C t Cj). 

ieOfc 



Since the matrix C is a uniform tight frame, for any t £ Oj, the matrix 
Cj Ct can be written as (m/iV)[I + At], where the (A; x k) matrix At is 



defined in ( 15 ). Similar to the worst-case design, we can derive the following 
inequality: 



A mi n(CfCt)>(-)(l- Y. l COS «Mj) 



teOfc 



>-^ 



fJ-C) 



,171, 



(^)(l-(k(k-l)/2)nc). 
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Taking the expectation, we get 



E Tfc A min (C^C Tfc ) > p Y, (^)(1 - (Hk - 1)/2) MC ) 

= P ( N k y^)(i-(k(k-i)/2) f , C ) 

Tf) 

= Q(l-(k(k-l)/2)n c ). 
This completes the proof. □ 
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